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ABSTRACT:  The  Virtual  Employment  Test  bed  (VETB)  allows  engineers  to  test  and  evaluate  material  solutions  in  a 
realistic  combination  live/virtual  environment,  as  well  as  the  associated  user  performance  characteristics.  This 
program  brings  together  current,  state-of-the-art,  ARDEC  capabilities  to  form  a  user  performance  evaluation  test  bed 
where  ARDEC  designs  will  be  evaluated  under  operational  scenarios  at  any  point  in  the  system  lifecycle. 


1.  Introduction 

There  is  always  a  desire  to  gather  user  test  data  early  in 
the  product  development  life  cycle  in  order  to  avoid  the 
larger  costs  of  changing  a  design  later.  The  DoD  has 
encouraged  the  use  of  modeling  and  simulation  as  a 
way  of  achieving  early  insight  to  the  capabilities  or 
improvements  of  a  new  design. 

The  Virtual  Employment  Test  bed  (VETB)  allows 
engineers  to  test  and  evaluate  material 
solutions  in  a  realistic  combination  live/virtual 
environment,  as  well  as  the  associated  user 
performance  characteristics.  This  program  brings 
together  three  current,  state-of-the-art,  ARDEC 
capabilities  to  form  a  user  performance  evaluation  test 
bed  where  ARDEC  designs  will  be  evaluated  under 
operational  scenarios  at  any  point  in  the  system 
lifecycle. 

2.  Live  Virtual  Constructive  Testing  at  the 
Target  Behavioral  Response  Laboratory 

Since  its  inception  in  2004,  the  Target  Behavioral 
Response  Laboratory  (TBRL)  at  Picatinny  Arsenal  has 
conducted  comprehensive  Live- Virtual-Constructive 
(LVC)  studies  under  controlled  laboratory 
experimentation.  Construction  of  the  live  experiments 
is  grounded  in  behavioral  science.  Human  behavioral 
science  requires  close  attention  to  the  critical  issues 
inherent  in  human  experimentation,  such  as  internal 
and  external  validity,  experimental  controls,  subject 


selection  and  protection  issues,  and  statistical  power 
(Cooper  2012). 

Behavioral  scientists  strive  to  configure  the  laboratory 
situation  to  systematically  manipulate  and  capture  the 
most  relevant  and  fundamental  factors  determining 
behavior  of  the  subjects.  This  task  involves 
constructive  simulations  of  the  external  environment  of 
physical  structures  and  spaces. 

To  create  an  operational  environment,  the  capabilities 
of  the  Gaming  &  Interactive  Technologies  Multimedia 
(GITM)  group  were  used  to  fashion  an  immersive 
synthetic  environment.  This  environment  is  used  to 
create  scenarios  that  test  a  design  with  respect  to  the 
users’  requirements. 

3.  The  Virtual  Employment  Test  Bed 

The  Virtual  Employment  Test  Bed  (VETB)  provides  a 
180  degree  wrap  around  environment  where  any 
ARDEC  (and  PM)  system  can  be  evaluated.  VETB 
provides  engineer  and  user  jury  feedback,  in  an 
operational  environment  to  capture  more  realistic 
performance  measurements.  Once  users  have 
completed  an  exercise,  their  performance  data,  such  as 
accuracy  and  reaction  time,  will  be  immediately 
available.  System  adjustments  can  be  made  and  tests 
rerun  rapidly.  The  VETB  offers  flexibility  and  can  be 
altered  for  specific  systems  if  necessary. 
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While  the  typical  first  person  shoot  simulations  are 
usually  used  for  training,  the  TBRL  utilizes  virtual 
adversaries  to  conduct  testing  of  the  design  and 
function  of  equipment.  The  VETB  is  an  integration  of 
several  systems  that  creates  a  full  scale  targeting 
measurement  system.  The  VETB  is  configured  for 
data  collection,  including  video  recording  equipment 
and  motion  capture  cameras  and  sensors  on  guns  to 
record  trigger  pulls.  The  test  bed  also  includes 
computer  control  systems,  video  recording,  motion 
capture  (MOCAP)  (Figure  4.1),  audio/visual 
equipment,  and  flexible  integration  of  stimulus  devices; 
x,  y,  z  location  coordinates  are  captured  in  real  time. 
This  real  time  information  allows  the  TBRL  to  test 
system  characteristics.  These  methods  also  allow  the 
TBRL  to  test  the  impact  of  platform  characteristics  on 
weapon  effectiveness,  for  example  slew  rate,  speed  of 
targeting,  rate  of  fire,  etc.  This  capability  is  most 
useful  in  examining  the  parameters  determining  the 
effectiveness  of  weapons  fire. 

In  the  current  configuration  of  the  test  bed  and 
experimental  design,  the  adversary  placement  is 
designed  to  evaluate  the  impact  of  changes  to  the 
Gunner  Protection  Kit  (GPK)  on  targeting  for  convoy 
protection  (Figure  4.1).  The  test  bed  consists  of  a  wrap 
around  screen  that  displays  an  immersive  game 
simulation  (Figure  4.3).  Subjects  are  placed  in  the 
GPK;  both  person  and  gun  are  outfitted  with  motion 
capture  sensors  (Figure  4.4).  Paintball  guns  fire  at  the 
subject  from  angles  that  correspond  to  the  adversary 
fire  displayed  on  the  screen.  The  adversaries  are 
positioned  onscreen  so  that  the  subject  will  be  required 
to  swing  the  gun  through  a  spectrum  of  angles  and 
speeds  in  order  to  accurately  aim  at  adversaries  in  the 
simulation.  The  set-up  is  instrumented  to  record  time 
to  acquire  target,  time  to  get  gun  on  target,  accuracy  of 
aim,  number  of  bullets  expended,  latency  to  fire,  and 
time  to  acquire  next  target.  The  combination  of  live 
subjects  operating  a  real  GPK  and  a  50-caliber  gun 
shooting  air  bullets  against  Virtual  Adversaries  allows 
testing  of  the  full  range  of  targeting  functions  and 
weapon  performance. 


Figure  4.1:  The  Virtual  Employment  Test  Bed  with  the 
GPK.  The  glowing  spheres  allow  motion  capture  of  subject 
and  gun  movement  by  the  cameras  mounted  on  the  tops  of 
the  wall. 


Figure  4.2:  Motion  capture  coordinates. 


5.  VETB  and  Decision  Support 

The  VETB  experiment  is  constructed  to  allow  TBRL  to 
collect  data  that  is  analyzable — including  experimental 
control  over  conditions,  same  comparable  conditions, 
repeated  measures,  number  of  subjects  with  sufficient 
statistical  power,  and  comparison  baseline  conditions. 
The  LVC  data  collected  allow  for  statistical  analyses 
and  data-based  decisions  for  acquisition,  training, 
doctrine,  and  commander  decisions.  Standard 
benchmarks  can  be  created  and  alternative  options  can 
be  compared  using  numbers  and  statistics.  These 
quantitative  analyses  can  then  serve  as  a  basis  for 
support  for  decisions  across  DoD.  The  production  of 
quantitative  data  for  appropriate  decision-making  is  the 
greatest  benefit  for  the  use  of  LVC  methods  for  testing. 
The  VETB  data  also  can  be  used  in  developing 
decision  support  modeling  and  simulation  tools 
(Mezzacappa,  et  al,  2012;  Riedener  &  Mezzacappa, 
2012). 
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Figure  4.3  The  Virtual  Employment  Test  Bed  with  the  GPK.  Subjects  target  adversaries  displayed  on  video  projections 
upon  the  wrap  around  screens.  Trigger  pulls  and  aim  on  the  screen  are  recorded.  Live  fire  paintball  guns  are  mounted 
on  top  of  the  walls.  Picture  by  Todd  Mozes. 


Figure  4.4:  Motion  capture  coordinates  overlaid  over  a  video  recording  of  a  gunner  in  the  GPK. 
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